Repository navigation
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
github-actions
Bot
added
the
size/normal
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
…l chain
Record the endpoints source commit in every run's report. resolve_git_sha
falls through live git of the package's own source checkout (dev,
bind-mounted sources) -> baked build-info (wheels) -> ENDPOINTS_GIT_SHA
(containers) -> none, validating each channel against a hex-SHA grammar
([0-9a-fA-F]{7,64}, optional -dirty) and appending -dirty for uncommitted
tracked changes. The report now always prints
`Git SHA: <sha> (source: <channel>)` and records git_sha_source in
result_summary.json.
- utils/version.py: _valid_sha grammar on all channels; git is consulted only
from the <root>/src/inference_endpoint/ layout, behind a toplevel guard;
git-location env family scrub (GIT_DIR/GIT_WORK_TREE/GIT_INDEX_FILE/...);
fail-to-dirty `git status --porcelain --untracked-files=no` probe under
GIT_OPTIONAL_LOCKS=0 (copied/rsynced checkouts aren't falsely dirty, the
index is never written); corrupt-bake tolerance; resolved once per report
(no lru_cache)
- metrics/report.py: new required git_sha_source field; unconditional Git SHA
display line
- scripts/write_build_info.py: build-time generator (toplevel + hex validated,
own env scrub, same dirty probe, atomic write); .dockerignore keeps a stray
local bake out of images; uv_build packs it into wheels
- scripts/Dockerfile.dev: ARG GIT_SHA -> ENV ENDPOINTS_GIT_SHA, declared last
so the per-commit value doesn't bust cached layers
- scripts/push_docker_image.sh: forward GIT_SHA (+ -dirty) so published images
report their commit at runtime
- docs/metrics/*, docs/utils/DESIGN.md: document the field + fallback chain
- tests: version, generator, report serialization
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
arekay-nv
force-pushed
the
feat/git-sha-provenance
branch
from
October 24, 2026 18:29
88625c9 to
1227aec
Compare
9 tasks done
… setup Address review findings on the git SHA provenance chain: - Drop the baked _build_info.py channel (generator, its tests, .dockerignore, .gitignore entry); it returns together with the CI step that produces it. The chain is now git -> env -> none, and ENDPOINTS_GIT_SHA is ignored when the package sits in a git checkout that git can't read, so a bind-mounted checkout never reports an image's SHA for other code. - Scope the dirty probe to the code that runs and its pins (src/inference_endpoint, pyproject.toml, uv.lock, the VBench scorer subproject) and run it with core.checkStat=minimal, so LFS data and synthetic inode numbers (Docker Desktop bind mounts) no longer read as -dirty. A failed probe still reports -dirty and now logs why; the probe budget is 5s. - Resolve provenance once per process: setup_benchmark calls get_version_info() first and logs it, so main-run and audit-phase reports agree and no git subprocess runs after the benchmark. - Read rev-parse output as bytes so a non-UTF-8 checkout path can't fail report building. - push_docker_image.sh / Dockerfile.dev: --short=7 for the image tag and GIT_SHA, dirty check with the runtime's pathspec. - Tests: hermetic git helpers, a parametrized resolver table, and real-git tests for dirty scope, copied checkouts, filtered files with stat-only changes, unreadable checkouts and index immutability, plus a test keeping the image-build pathspec in sync with the runtime's. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Records the endpoints source commit in every run's report, so a
result_summary.json/report.txtis self-describing about which codeproduced it.
resolve_git_sha()walks a channel chain (first hit wins), validating everyvalue against a hex-SHA grammar (
[0-9a-fA-F]{7,64}, optional-dirty):gitgit rev-parseof the package's own source checkout (src/layout + toplevel guard, not cwd)envENDPOINTS_GIT_SHA(docker--build-arg GIT_SHA→ENV); ignored inside a git checkoutnoneA checkout reports live git or nothing. If
<root>/.gitexists but git can'tread it (git missing, a timeout, dubious ownership, a
.gitpointing outside abind mount), the report says
unknown (source: none)and a warning is logged,rather than recording an image's
ENDPOINTS_GIT_SHA, which describes othercode.
The SHA gains
-dirtywhen tracked files in the code that runs differ fromHEAD:src/inference_endpoint,pyproject.toml,uv.lock, and the VBenchscorer subproject under
examples/. Untracked files and LFS data elsewhere areignored. The probe is
git -c core.checkStat=minimal status --porcelain --untracked-files=no -- <paths>under
GIT_OPTIONAL_LOCKS=0: copied/rsynced checkouts and Docker Desktop bindmounts (synthetic inode numbers) aren't re-hashed, so they aren't falsely dirty,
and the caller's index is never written. A probe that fails or times out (5s)
reports
-dirtyand logs why.Provenance is resolved once per process, at the start of
setup_benchmark(and logged), so every report the process writes — main run and audit phases —
records the same SHA, sampled before the run rather than after it.
The report always prints
Git SHA: <sha> (source: <channel>), andgit_sha_sourceis recorded inresult_summary.json.Changes
utils/version.py: validated git → env → none resolver; checkout gate;scoped
checkStat=minimaldirty probe that warns on failure; bytes-saferev-parse; git repo-location env scrub (GIT_DIR/GIT_WORK_TREE/GIT_INDEX_FILE/…) so a strayGIT_DIRcan't slip a foreign SHA past thetoplevel guard; per-process memo.
commands/benchmark/execute.py:setup_benchmarkpins and logsprovenance before any work runs.
metrics/report.py: new requiredgit_sha_sourcefield; unconditionalGit SHA:display line.scripts/push_docker_image.sh(feat :Testing workflow to push images #405 publish path): forwards--build-arg GIT_SHA=<short sha>[-dirty]— the same--short=7SHA as theimage tag, with the runtime's dirty scope — so published GHCR images report
their commit at runtime instead of
source: none.scripts/Dockerfile.dev:ARG GIT_SHA→ENV ENDPOINTS_GIT_SHA, last sothe per-commit value doesn't bust cached layers; documented build command
carries
-dirty.docs/metrics/*,docs/utils/DESIGN.md) + tests.Testing
test_version.py: hermetic tmp repos (no ambient git config orGIT_*);real-git tests for dirty scope, copied checkouts, filtered (LFS-style) files
with stat-only changes, unreadable checkouts (gitfile outside a mount,
dangling
.gitsymlink) and index immutability; fake-subprocess.runteststhat decode exactly when the real call would; a test keeping the image-build
pathspec equal to the runtime's. Mutation-checked: reverting any of the
behaviors above fails at least one test.
inference-endpoint-dev(no git-lfs) and the LFS parquetsmudged in a clean bind-mounted clone:
(sha, "git"), no false-dirty; abind-mounted worktree whose
.gitpoints outside the mount:(None, "none")plus the warning, with the image's
ENDPOINTS_GIT_SHAset.from-configwith anaudit:output-caching block): the setup log line precedes the run;report.txtandresult_summary.jsoncarry the same SHA across the main andaudit-phase reports; git runs only at setup (2 invocations per process).
main(remaining local failures are themacOS-only socket-path /
sched_*affinity//procones, identical onmain).Follow-ups (not in this PR)
the CI step that generates it (and removes it after
uv build).Notes
Report.git_sha_sourceis a new required struct field; nothing decodesReportback from JSON, so there is no wire/back-compat break..git(atarball, or
src/alone) can't be told apart from the image's own copy andreports the image's
ENDPOINTS_GIT_SHA. Mount the whole checkout.🤖 Generated with Claude Code